Providing synchronous content and supplemental experiences

ABSTRACT

Methods, systems, computer readable media, and apparatuses for providing synchronous supplemental experiences are presented. According to one or more aspects, a video signal may be transmitted to a display device, and a sync signal may be transmitted to at least one other device. The sync signal may include an identifier of a program currently being displayed and a time value indicating a current point in time of the program. In at least one arrangement, the sync signal may cause the at least one other device to access content synchronization data; determine, based on the content synchronization data, that at least one supplemental experience corresponds to the identifier of the program and the time value; and present the at least one supplemental experience to a user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/125,144, filed Sep. 7, 2018, which is a continuation of U.S.application Ser. No. 13/313,531, filed Dec. 7, 2011 (now U.S. Pat. No.10,142,121). The above-referenced applications are hereby incorporatedby reference in their entirety.

BACKGROUND

Increasingly, many people are coming to own and operate multiplecomputing devices of various kinds in their homes, workplaces, and otherenvironments. In an example home, a family may have a computing devicethat receives digital audiovisual content, another computing device thatthey use primarily for browsing a network such as the Internet andviewing digital data, and still another computing device, such as alaptop computer or a smartphone, that they use for word processing,email communications, playing video games, and so on. This disclosureprovides aspects for more convenient, functional, and useable ways ofproviding synchronized content experiences across multiple computingdevices.

SUMMARY

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the disclosure. The summary is not anextensive overview of the disclosure. It is neither intended to identifykey or critical elements of the disclosure nor to delineate the scope ofthe disclosure. The following summary merely presents some concepts ofthe disclosure in a simplified form as a prelude to the descriptionbelow.

Some aspects of this disclosure relate to providing synchronized contentexperiences across multiple computing devices, and in particularaspects, to providing synchronous supplemental experiences. Forinstance, according to one or more aspects of the disclosure, a contentand/or service provider(s) may be able to provide users with enhanceduser experiences that take advantage of the user having additionalcomputing devices in their local area beyond a traditional contentreceiver device. For example, in addition to having a content receiverdevice and a display device connected thereto for displaying receivedvideo programming, the user also may have a tablet computer or asmartphone that the user might use while simultaneously viewing contentthat is received by the content receiver device. By implementing one ormore aspects of the disclosure, supplemental content related to thevideo content being received by the content receiver and displayed onthe display device, for instance, may be presented on the user's tabletcomputer, smartphone, etc., and the presentation of this supplementalcontent may be synchronized with the display of the video content.

According to one or more aspects, a video signal may be transmitted to adisplay device. Simultaneously, or shortly thereafter, a synchronizationsignal may be transmitted to at least one local device, and thesynchronization signal may include an identifier of a program currentlybeing displayed and a time value indicating a current point in time ofthe program. In one or more arrangements, the synchronization signal maycause the at least one local device to access a content synchronizationmap; determine, based on the content synchronization map, that at leastone supplemental experience corresponds to the identifier of the programand the time value; and provide access to the at least one supplementalexperience.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIG. 1 illustrates an example information distribution network accordingto one or more aspects described herein.

FIG. 2 illustrates an example hardware platform on which the variouselements described herein may be implemented according to one or moreaspects described herein.

FIG. 3 illustrates an example method of creating content synchronizationdata according to one or more aspects described herein.

FIG. 4 illustrates an example method of transmitting a synchronizationsignal according to one or more aspects described herein.

FIG. 5 illustrates an example method of processing a synchronizationsignal according to one or more aspects described herein.

FIG. 6 illustrates example user interfaces by which a supplementalexperience may be provided according to one or more aspects describedherein.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which aspects of the disclosure may be practiced. It isto be understood that other embodiments may be utilized, and structuraland functional modifications may be made, without departing from thescope of the present disclosure.

FIG. 1 illustrates an example information distribution network 100 inwhich many of the various features described herein may be implemented.Network 100 may be any type of information distribution network, such assatellite, telephone, cellular, wireless, etc. One example may be anoptical fiber network, a coaxial cable network, or a hybrid fiber/coaxdistribution network. Such networks 100 use a series of interconnectedcommunication links or lines 101 (e.g., coaxial cables, optical fibers,wireless, etc.) to connect multiple premises 102 (e.g., businesses,homes, consumer dwellings, etc.) to a central office or headend 103. Thecentral office 103 may transmit downstream information signals onto thelines 101, and each home 102 may have a receiver used to receive andprocess those signals.

There may be one line 101 originating from the central office 103, andit may be split a number of times to distribute the signal to varioushomes 102 in the vicinity (which may be many miles) of the centraloffice 103. The lines 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to help convey the signalclearly, but in general each split introduces a bit of signaldegradation. Portions of the lines 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths.

The central office 103 may include an interface or termination system(TS) 104, such as a cable modem termination system (CMTS), which may bea computing device configured to manage communications between deviceson the network of lines 101 and backend devices such as servers 105-107(to be discussed further below). The TS may be as specified in astandard, such as the Data Over Cable Service Interface Specification(DOCSIS) standard, published by Cable Television Laboratories, Inc.(a.k.a. CableLabs), or it may be a similar or modified device instead.The TS may be configured to place data on one or more downstreamfrequencies to be received by modems at the various homes 102, and toreceive upstream communications from those modems on one or moreupstream frequencies. The central office 103 may also include one ormore network interfaces 108, which can permit the central office 103 tocommunicate with various other external networks 109. These networks 109may include, for example, networks of Internet devices, telephonenetworks, cellular telephone networks, fiber optic networks, localwireless networks (e.g., WiMAX), satellite networks, and any otherdesired network, and the interface 108 may include the correspondingcircuitry needed to communicate on the network 109, and to other deviceson the network such as a cellular telephone network and itscorresponding cell phones.

As noted above, the central office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the central office 103 may include a push notification server105. The push notification server 105 may generate push notifications todeliver data and/or commands to the various homes 102 in the network (ormore specifically, to the devices in the homes 102 that are configuredto detect such notifications). The central office 103 may also include acontent server 106. The content server 106 may be one or more computingdevices that are configured to provide content to users in the homes.This content may be, for example, video on demand movies, televisionprograms, songs, text listings, etc. The content server 106 may includesoftware to validate user identities and entitlements, locate andretrieve requested content, encrypt the content, and initiate delivery(e.g., streaming) of the content to the requesting user and/or device.

The central office 103 may also include one or more application servers107. An application server 107 may be a computing device configured tooffer any desired service, and may run various languages and operatingsystems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD,Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, anapplication server may be responsible for collecting television programlistings information and generating a data download for electronicprogram guide listings. Another application server may be responsiblefor monitoring user viewing habits and collecting that information foruse in selecting advertisements. Another application server may beresponsible for formatting and inserting advertisements in a videostream being transmitted to the homes 102.

An example premises 102 a, such as a home, may include an interface 120.In one embodiment, the interface may comprise a modem 110, which mayinclude transmitters and receivers used to communicate on the lines 101and with the central office 103. The modem 110 may be, for example, acoaxial cable modem (for coaxial cable lines 101), a fiber interfacenode (for fiber optic lines 101), or any other desired modem device. Themodem 110 may be connected to, or be a part of, a gateway interfacedevice 111. The gateway interface device 111 may be a computing devicethat communicates with the modem 110 to allow one or more other devicesin the home to communicate with the central office 103 and other devicesbeyond the central office. The gateway 111 may be a set-top box (STB),digital video recorder (DVR), computer server, or any other desiredcomputing device. The gateway 111 may also include (not shown) localnetwork interfaces to provide communication signals to devices in thehome, such as televisions 112, additional STBs 113, personal computers114, laptop computers 115, wireless devices 116 (wireless laptops andnetbooks, mobile phones, mobile televisions, personal digital assistants(PDA), etc.), and any other desired devices. Examples of the localnetwork interfaces include Multimedia Over Coax Alliance (MoCA)interfaces, Ethernet interfaces, universal serial bus (USB) interfaces,wireless interfaces (e.g., IEEE 802.11), Bluetooth interfaces, andothers.

FIG. 2 illustrates general hardware elements that can be used toimplement any of the various computing devices discussed above. Thecomputing device 200 may include one or more processors 201, which mayexecute instructions of a computer program to perform any of thefeatures described herein. The instructions may be stored in any type ofcomputer-readable medium or memory, to configure the operation of theprocessor 201. For example, instructions may be stored in a read-onlymemory (ROM) 202, random access memory (RAM) 203, removable media 204,such as a Universal Serial Bus (USB) drive, compact disk (CD) or digitalversatile disk (DVD), floppy disk drive, or any other desired electronicstorage medium. Instructions may also be stored in an attached (orinternal) hard drive 205. The computing device 200 may include one ormore output devices, such as a display 206 (or an external television),and may include one or more output device controllers 207, such as avideo processor. There may also be one or more user input devices 208,such as a remote control, keyboard, mouse, touch screen, microphone,etc. The computing device 200 may also include one or more networkinterfaces, such as input/output circuits 209 (such as a network card)to communicate with an external network 210. The network interface maybe a wired interface, wireless interface, or a combination of the two.In some embodiments, the interface 209 may include a modem (e.g., acable modem), and network 210 may include the communication lines 101discussed above, the external network 109, an in-home network, aprovider's wireless, coaxial, fiber, or hybrid fiber/coaxialdistribution system (e.g., a DOCSIS network), or any other desirednetwork.

According to one or more aspects, various entities and/or devices mayperform methods (e.g., steps) in providing synchronous supplementalexperiences to one or more users (e.g., content viewers). The flowchartsshown in FIGS. 3-5 illustrate some examples of these methods, which aredescribed in greater detail below. As an overview, FIG. 3 illustrates anexample method that may be performed by a content creator, for instance,in creating content (e.g., television programming) and in creating oneor more corresponding supplemental user experiences. FIG. 4 illustratesan example method that may be performed by a device that receives,decodes, and/or displays such content (e.g., gateway 111, television112, set top box 113, personal computer 114, etc.), where, for instance,the device communicates with another device on which one or moresupplemental experiences are presented. FIG. 5 illustrates an examplemethod that may be performed by a device on which one or moresupplemental experiences are presented (e.g., laptop computer 115,wireless device 116, etc.). Each of these example methods will now beindividually described.

FIG. 3 illustrates an example method of creating content synchronizationdata, such as a map, according to one or more aspects described herein.According to one or more aspects, any and/or all of the methods andmethod steps described herein may be performed by a computing device(e.g., application server 107, gateway 111, personal computer 114,computing device 200, etc.), embodied in computer-executableinstructions stored on one or more non-transitory computer-readablemedia, and/or embodied in computer-readable instructions stored in thememory of an apparatus that includes at least one processor which mayexecute the instructions.

In step 301, a content creator, provider, or producer may create programcontent (a network may create content or an individual may uploadcontent to a computing device for sharing, for example). The programcontent then may be received and/or stored by a computing device (e.g.,a personal computer used by the content creator, a server accessible tothe content creator, etc.) and/or stored on one or morecomputer-readable media. For example, a television producer may create(e.g., write, film, edit, etc.) a television program and may store thetelevision program (e.g., the digital video and audio data making up thetelevision program) on a computing device. In another example, a movieproducer may create (e.g., write, film, edit, etc.) a movie and maysimilarly store the movie (e.g., the digital video and audio data makingup the movie) on a computing device.

In step 302, the content creator (or another party) may identify pointsin the program content at which one or more supplemental experiencesshould be added and/or defined. The content creator's computing devicethen may store information (e.g., in a data table) indicating whichsupplemental experience(s) should occur at what point(s) of the program.For example, the stored information may include an Internet UniformResource Locator (URL), or the name of a locally-stored computersoftware routine to be executed. The content creator's computing devicealso may store additional information about the supplemental experiences(e.g., any related computer-readable instructions, graphics, audioand/or video content, etc.) which may be used by a user device inproviding one or more supplemental experiences, as further describedbelow. In some arrangements, the content creator's computing device mayautomatically identify points in the program content at which one ormore supplemental experience should be added and/or defined, forinstance, by using video analysis software and/or algorithms (e.g.,which may identify points in the program where credits start rolling, aparticular actor or actress begins speaking, a commercial break startsor ends, other changes in the audio and/or video feed, etc.).

Supplemental experiences may generally include interactive and/ornon-interactive content that is displayed and/or otherwise presentedindependently of related primary content. Primary content may include,for instance, content such as television programming, movies, data, andso on, that is complemented by the content making up the correspondingsupplemental experience. In some arrangements, independently presentinga supplemental experience may include displaying and/or playing theinteractive and/or non-interactive content making up the supplementalexperience on a device different from the device on which correspondingprimary content is displayed and/or presented. For instance, primarycontent, such as a television show, may be displayed on display 112,while a supplemental experience related to the primary content, such asan interactive trivia game related to the television show, may bedisplayed, executed, and/or otherwise presented on a viewer's laptop ortablet computer (e.g., laptop computer 115, wireless device 116, etc.).In other arrangements, independently presenting a supplementalexperience may include displaying and/or playing the interactive and/ornon-interactive content making up the supplemental experience on thesame device upon which corresponding primary content is displayed, butin a different screen area (e.g., a dedicated window, an overlay, etc.).

Thus, in step 302, a producer of television programming, for instance,may identify points in the television program at which one or moresupplemental experiences should be added and/or defined. Alternatively,such identification may be done by a different party. In manyarrangements, the point(s) of the program at which supplementalexperience(s) are to be defined may be selected such that these pointscorrespond to the content being displayed and/or otherwise played atthat point in time of the program. For example, if the televisionprogram is a game show with three rounds (e.g., a first round, a secondround, and a final round), a first supplemental experience may becreated and defined to begin and coincide with playback of the firstround of the game show, a second supplemental experience (which may bedifferent from the first supplemental experience) may be created anddefined to begin and coincide with playback of the second round of thegame show (and optionally cause playback of the first supplementalexperience to end), and a third supplemental experience (which may bedifferent from the first supplemental experience and the secondsupplemental experience) may be created and defined to begin andcoincide with playback of the final round of the game show (andoptionally cause playback of the first and/or second supplementalexperience to end). In another example, where the program is a featurefilm or movie, supplemental experiences may be created and defined tobegin and coincide with particular events in the plot of the featurefilm or movie.

Additionally or alternatively, supplemental experiences may be createdand defined to begin and coincide with particular playback events, suchas fast-forward and/or rewind operations, sound muting operations, andso on. For example, a first supplemental experience (e.g., displayingplot summary, program navigation information, and a first set ofadvertising content) may be created and defined to begin and coincidewith fast-forward operations, while a second supplemental experience(e.g., displaying a transcript of the program's audio content and asecond set of advertising content) may be created and defined to beginand coincide with sound muting operations.

Additionally or alternatively, one or more supplemental experiences maybe synchronized with primary content such that content associated with asupplemental experience is presented to a viewer at the same time inrelation to the primary content regardless of whether the viewer iswatching the primary content live or in a time-shifted context. In atleast one arrangement, in a time-shifted context, some aspects of thecontent associated with the supplemental experience may be impacted byactions made by other viewers when the other viewers experienced suchcontent. For example, if multiple viewers (e.g., 100 people) watch aparticular program live (e.g., as the particular program is firsttransmitted, e.g., broadcast, multicast, unicast, etc.), these viewersmay interact with a supplemental experience that includes aprogram-related poll (e.g., by voting) and/or with another supplementalexperience that aggregates program-related comments posted by viewers ona message service (e.g., a social networking service, such as TWITTER).As viewers vote or post comments during the timeline of the program, acomputing device (e.g., an application server) may, for instance, trackand record the times at which particular votes were provided and/or atwhich particular comments were made. Subsequently, when another viewerbegins watching the same program the next day (or in any othertime-shifted context), that viewer, when being presented with thesupplemental experience, may be presented with (and/or otherwise view)the previously casted votes and/or the previously posted comments at thesame time that such votes and/or comments were originally cast and/ormade in relation to the program. In this way, viewers watchingprogramming in a time-shifted context may feel as though they areexperiencing the supplemental experience in real-time (e.g.,synchronized with the live transmission and/or broadcast of theprogram), even though these viewers may actually be interacting with thecontent hours, days, or weeks after the primary content was first madeavailable. While poll votes and message board comments are used indescribing this example, any kind of user-generated and/oruser-submitted content could similarly be captured and later presentedto viewers viewing content in a time-shifted context.

Subsequently, in step 303, the content creator's one or more computingdevices may generate content synchronization data, such as a map. Thecontent synchronization map may be generated based on the pointsidentified in step 302 at which supplemental experiences are to occur.In addition, the content synchronization map may take the form of a datatable in which may be stored the one or more time values at whichparticular supplemental experience(s) are to be displayed. In somearrangements, the content synchronization map (e.g., the data table)also may store the information making up the supplemental experiences(e.g., video content, audio content, graphical content, interactivecontent and/or program code for interactive applications, etc.), whilein other arrangements, the content synchronization map (e.g., the datatable) may store pointers and/or references to the information making upthe supplemental experiences (e.g., because such information may bestored in one or more locally stored and/or remotely accessible files).In at least one arrangement, the information making up the supplementalexperiences may be centrally stored on one or more web servers, suchthat this information may be more easily updated and centrallymaintained, and one or more devices that display and/or otherwise playback such supplemental experiences may download the information via theInternet. An example map table will be discussed further below inconnection with FIG. 5 .

In step 304, the content creator's computing device may deliver theprogram content and the content synchronization map. For example, instep 304, the content creator's computing device may deliver the programcontent and the content synchronization map by transmitting the programcontent to a service provider's computing device (e.g., a headend orother computing device at central office 103) and publishing the contentsynchronization map to a web server (e.g., application server 107). Insome arrangements, the content creator may publish the contentsynchronization map to a web server that they (the content creator) ownand/or operate, while in other arrangements, the content creator maypublish the content synchronization map to a web server that the serviceprovider owns and operates.

As described in greater detail below, by delivering the program contentand the content synchronization map in this way, the content creator mayallow one or more users (e.g., content viewers) to view the deliveredcontent and be presented with one or more supplemental experiences, asdefined by the content synchronization map, for instance. Havingdescribed the method of FIG. 3 , which illustrates what a contentcreator (and/or the content creator's computing device) might do tocreate content and one or more corresponding supplemental experiences,the method of FIG. 4 , which illustrates what a content receiver (e.g.,a user device, such as a gateway 111 provided to a user by a serviceprovider) might do in receiving such content, will now be described.

FIG. 4 illustrates an example method of transmitting a synchronizationsignal according to one or more aspects described herein. According toone or more aspects, the method illustrated in FIG. 4 may be performedby a device that receives, decodes, and/or displays content to a user(e.g., a computing device, gateway 111, display device 112, set top box113, personal computer 114, a smartphone, etc.). The device may belocated at a provider location or a user location, such as premises 102a, and, as further described below, the device may communicate withanother device at the user location on which one or more supplementalexperiences are presented.

In step 401, a terminal such as a content receiver device may, ifneeded, initialize and boot up, e.g., by loading into memory operatingsystem software and/or other programs that allow the device to performvarious functions, such as those described below.

In step 402, the content receiver device may pair with another device,such as a user's mobile device on which one or more supplementalexperiences are to be presented. The user's mobile device may be, forinstance, a smart phone (e.g., an APPLE iPhone), a laptop computer, acomputing tablet (e.g., an APPLE iPad), and/or the like. According toone or more aspects, pairing the content receiver with the user's mobiledevice may involve establishing a data connection between the twodevices via a local network, such as a wireless local area network(WLAN) at the user location, and/or via other connection mechanisms,such as via a BLUETOOTH connection. In other arrangements, such a dataconnection may be established over a wide area network, such as theinternet, and/or may leverage two-way data communications over cableinfrastructure. In still other arrangements, the user's mobile devicemay be paired with the content receiver by way of an audiosynchronization signal emitted by the content receiver (e.g., at aparticular frequency) and received by the user's mobile device (e.g.,via a microphone included in the device). Although the word “pair” isused here to describe the connection between the content receiver deviceand the user's mobile device, it should be understood that the contentreceiver device may simultaneously be paired to a plurality of userdevices.

In some arrangements, pairing may be done anonymously, such that a usermight not need to create an account, login, or any other sort ofidentity in order to pair the user's mobile device with the contentreceiver. In other arrangements, pairing may leverage new or existinguser accounts, such as one or more accounts the user may maintain withone or more social networks (e.g., FACEBOOK, TWITTER, etc.) and/or anyother services.

Once the content receiver device (or another type of device) has pairedwith the user's mobile device, for instance, in step 403, the contentreceiver device may determine whether the user is currently playing,viewing, or otherwise consuming a program. For example, in step 403, thecontent receiver device may determine whether it is displaying, orcausing to be displayed, content (e.g., audiovisual content, such aslinear or on-demand video programming) to the user (as opposed todisplaying a full-screen menu that does not include any audiovisualprogramming content, for instance).

Subsequently, if the content receiver device determines, in step 403,that the user is currently consuming content, such as a program, then instep 404, the content receiver device may identify the program beingplayed. In identifying the program being played, the content receiverdevice may, for example, extract and store information about the programfrom headers of data packets that include the audiovisual data of theprogram and that are received by the content receiver device, and/or thecontent receiver device may access, retrieve, and store informationabout the program from one or more data tables, such as data tablesincluded in a database stored on the service provider's server (e.g.,content server 106, application server 107, etc.). The programinformation stored by the content receiver device in this step mayinclude the title of the program, the genre of the program, a uniqueidentifier (e.g., identification number or string of alphanumericcharacters) corresponding to the program, the length of the program(e.g., in number of frames, units of time, and/or any other desiredunits), a channel number corresponding to the channel on which theprogram is being received (if the program is being received on aparticular channel), the name of the channel on which the program isbeing received (again, if the program is being received on a particularchannel), and/or a current playback position of the program (e.g., innumber of frames, units of time, and/or any other desired units).According to one or more aspects, the current playback position mayindicate how much of the program has elapsed as of the current time. Forinstance, the current playback position may be expressed as frame number27,456 of 54,000, or 50.84% elapsed, or 915.2 seconds elapsed. Accordingto one or more additional aspects, the current playback position mayinclude, instead of or in addition to a current timestamp, a uniquemarker or trigger (e.g., point A, point B, point C, etc.) that indicatesa position in the timeline of the program.

In addition to identifying information about the program, the contentreceiver may also gather information about device capabilities (e.g.,functions that the content receiver is capable of, such as whether thecontent receiver includes a digital video recorder (DVR) function and/orwhether the content receiver can receive and display high definition(HD) content, etc.) and/or user account information (e.g., subscriberinformation, such as what channels and/or packages the user mightreceive, what channels the user has access to, what other content theuser may be entitled to view, what other services the user might receivethrough the service provider, such as digital voice and/or telephoneservices, etc.). Any and/or all of this information may, for instance,subsequently be transmitted to the user's mobile device, as furtherdescribed below, and may inform aspects of one or more supplementalexperiences displayed on the user's mobile device. For example, if aparticular viewer also receives digital voice and/or telephone services,a supplemental experience displayed during an advertisement mightinclude a button allowing a user to be directly connected to arepresentative of a company corresponding to the advertisement, so as toenable the viewer to purchase the company's goods and/or services beingadvertised. Additionally or alternatively, transmission of thisinformation may be controlled based on business rules, such as businessarrangements between, for instance, the content provider/distributor andthe content creator which developed a particular supplementalexperience.

After identifying the program currently being played in step 404, thecontent receiver device may, in step 405, determine the current playbackposition of the program. In one or more arrangements, the contentreceiver device may determine the current playback position of theprogram by referencing the program information identified in step 404,because as noted above, such program information may include informationspecifying the current playback position of the program. In one or morealternative arrangements, however, the content receiver device maydetermine the current playback position of the program based on thescheduled start time of the program (e.g., as indicated by and/orincluded in the program information identified in step 404) and thecurrent time. For instance, the content receiver device may subtract thescheduled start time of the program from the current time to obtain theamount of time that has elapsed in the program, and this elapsed timemay represent the current playback position.

Subsequently, in step 406, the content receiver device may transmitinformation to the user's mobile device (and/or to any other device thatwas paired with the content receiver device in step 402). Theinformation transmitted to the paired device (e.g., the user's mobiledevice) in this step may include current information about the programcurrently being played on and/or by the content receiver device, such asthe unique identifier of the program, the current playback position,and/or the like. Additionally or alternatively, the informationtransmitted to the paired device may include state information about thecontent receiver device, such as the current sound volume level, thecurrent playback speed, the current subtitle and/or caption settings,the current user permission level (e.g., specifying whether the user isallowed to access content of all rating levels or limited rating levelsbased on current parental control settings), and/or the like.

According to one or more aspects, the information transmitted in step406 may be sent as a data message to the user's mobile device. Inaddition, this message may constitute a “heartbeat” message, because thecontent receiver device may transmit such a message to the user's mobiledevice at a regular interval (e.g., every three seconds or at any otherdesired interval), as further described below.

In step 407, the content receiver device may wait for a predeterminedperiod of time. For example, the content receiver device may wait for anamount of time (e.g., fifty milliseconds, one or two seconds, etc.)before performing any further steps of the method illustrated in FIG. 4. The content receiver device may perform other actions during thistime, however, such as continuing to receive, decode, and displayaudiovisual content. By waiting for this predetermined period of time,the content receiver device may create the interval between heartbeatmessages sent by the content receiver device to the paired device. Inaddition, by increasing or decreasing the predetermined amount of timethat the content receiver device waits in step 407, the frequency atwhich the content receiver device sends such heartbeat messages to thepaired device may be correspondingly decreased or increased.

After the content receiver device has waited for the predeterminedperiod of time in step 407, the content receiver device may, in step408, determine whether the same program is still being played as whenthe previous heartbeat message was sent in step 406.

If the content receiver device determined, in step 408, that the sameprogram is still being played, then the content receiver device'sexecution of the method may return to step 405, where the contentreceiver device may determine the updated playback position of theprogram and subsequently transmit to the paired device another heartbeatmessage that includes updated playback position information. On theother hand, if the content receiver device determines, in step 408, thatthe program being played has changed since the previous heartbeatmessage was sent, then the content receiver device's execution of themethod may return to step 403, where the content receiver device maydetermine whether the user is currently playing a program.

By transmitting a synchronization signal, such as a heartbeat message,to a paired device in this way, the content receiver device may enablethe paired device (e.g., the user's mobile device) to know what programis currently being displayed and what part of the program is currentlybeing displayed by the content receiver device, among other things. Asdescribed below, this information may allow the paired to device topresent one or more supplemental experiences that may correspond toparticular points in the program. Having thus described the method ofFIG. 4 , which illustrates what a content receiver device might do totransmit a synchronization signal, the method of FIG. 5 , whichillustrates what a paired device, such as a user's mobile device, mightdo when receiving such a synchronization signal, will now be described.

FIG. 5 illustrates an example method of processing a synchronizationsignal according to one or more aspects described herein. According toone or more aspects, the method illustrated in FIG. 5 may be performedby a device that is paired to the content receiver device in theprevious method, such as a mobile device at the user's location that isused by the user (e.g., laptop computer 115, wireless device 116, etc.).In some embodiments, the method of FIG. 5 may be performed by thecontent receiver device itself or any other associated device thatprovides an additional user interface (e.g., a dedicated screen oroverlay in addition to the primary display/screen) for presenting asupplemental experience. For clarity, however, the steps of FIG. 5 aredescribed with regards to an example embodiment in which a mobile deviceis used.

In step 501, the user's mobile device may receive a request to enablepresentation of supplemental content. Such a request may be received,for instance, as user input provided via a user preferences or settingsmenu presented by the device. Additionally or alternatively, such arequest may correspond to the user launching a particular softwareapplication or performing another action on the mobile device. Thesoftware application may be developed and made available to the user bythe service provider and further may include program code that, whenexecuted, causes the user's mobile device to perform any and/or all ofthe steps of the method illustrated in FIG. 5 .

In step 502, the user's mobile device may connect to and/or be pairedwith another device, such as the content receiver device. For example,in step 502, the user's mobile device may establish a data connectionwith the content receiver device, as described above with respect tostep 402 of the method of FIG. 4 . In one or more arrangements, inaddition to receiving information from the content receiver device, suchas one or more heartbeat messages, the user's mobile device also maytransmit information to the content receiver device. For example, as oneor more supplemental experiences are presented on the user's mobiledevice based on the one or more heartbeat messages sent to the user'smobile device by the content receiver device, as further describedbelow, the user's mobile device may periodically send its ownsynchronization signal back to the content receiver device (e.g., areverse synchronization signal) indicating, for instance, whatsupplemental experience(s) is currently being presented. Such a reversesynchronization signal may, for instance, trigger changes on the primaryscreen (e.g., as displayed by or caused to be displayed by the contentreceiver device). For example, the reverse synchronization signal may beused to cause the content receiver device to change channels, launch anon demand content interface, load menus and/or applications, scheduleand/or play recordings, and/or change user settings. Additionally oralternatively, the reverse synchronization could be used in controllingon-demand or previously recorded content where the content beingdisplayed on the primary screen is impacted by viewer interactions withthe supplemental experience. For instance, if a supplemental experienceincludes a trivia game, then correct answers to questions in the triviagame might cause one overlay (e.g., “Good job! You answered correctly!”)to be displayed on the primary screen, while incorrect answers toquestions in the trivia game might cause a different overlay (e.g.,“Sorry, try again!”) to be displayed on the primary screen. In anotherexample, if a supplemental experience includes a fantasy sports game,then scoring updates in the fantasy sports game may cause notificationsto be displayed as overlays on the primary screen (e.g., “Score Update:Player AAA on Team BBB just scored a touchdown and gained you 7 points!Your current score is now 44 points!”).

Referring again to FIG. 5 , in step 503, it may be determined whether aheartbeat message has been received, e.g., by the user's mobile devicefrom the content receiver device. For example, in step 503, the user'smobile device may determine whether it has received a heartbeat messagefrom the content receiver device via the data connection established instep 502. As described in greater detail above, the heartbeat messagemay be a synchronization signal that includes information about aprogram currently being received, decoded, and/or displayed (or causedto be displayed) by the content receiver device.

If it is determined, in step 503, that a heartbeat message has not beenreceived, then in step 504, the user's mobile device may wait for apredetermined period of time, and then the method may return to step 503where it again may be determined whether a heartbeat message has beenreceived. For example, if no heartbeat message has yet been received,then the user's mobile device may wait for a short period of time (e.g.,one second, five seconds, etc.) and then may re-determine whether aheartbeat message has been received. This loop may continue until aheartbeat message is received, or alternatively, until the softwareapplication is exited and/or the user's mobile device is powered off.

If, on the other hand, it is determined in step 503 that a heartbeatmessage has been received, then in step 505, the user's computing devicemay determine whether the heartbeat message indicates that a new ordifferent program is now being displayed by the content receiver device.In determining whether the heartbeat message indicates that a new ordifferent program is being displayed by the content receiver device, theuser's mobile device may compare the program identifier and/or programtitle (or any other desired data) included in the heartbeat message withcorresponding information from a previously received heartbeat messageto determine whether the program has changed. Additionally oralternatively, if the heartbeat message is the first heartbeat messagereceived by the user's mobile device (e.g., because the device was justpowered on, the software application was just launched, etc.), then theuser's mobile device might not have information from a previousheartbeat message to compare the current heartbeat message to and thusmay determine that the heartbeat message indicates that a new ordifferent program is being displayed.

If it is determined, in step 505, that the heartbeat message indicatesthat a new or different program is now being displayed on the contentreceiver device, then in step 506, the user's mobile device may requestnew content synchronization data (e.g., a content synchronization map,similar to one generated in step 303 above) corresponding to the new ordifferent program. On the other hand, if the program has not changed andthe user's mobile device has already loaded the content synchronizationdata for the current program, then execution of the method may proceeddirectly to step 508, which is further described below.

According to one or more aspects, content synchronization data may bestored on a remote server (e.g., a web server), and the contentsynchronization data may be updated by a content creator (as describedabove, for instance) to include information about one or moresupplemental experiences corresponding to one or more particularprograms. Also, a different entity (e.g., a content provider, contentcreator, etc.) than the entity which creates or provides the content(e.g., program) may create or provide the content synchronization datacorresponding to that content. Further, multiple content providers orcontent creators may each provide or create portions of the contentand/or portions of the content synchronization data. For example, thecontent synchronization data stored on a remote server may be createdand maintained by multiple different entities. Moreover, in anotherexample, one entity may create one supplemental experience for specificcontent, while another entity may create another supplemental experiencefor the same specific content.

In one or more arrangements, content synchronization data may be a datatable in which information identifying one or more supplementalexperiences are stored in relation to one or more portions of a programand/or events. The following table illustrates an example of contentsynchronization data:

Supplemental Program Time Event Experience(s) Data Source Time:00:18:02.24- Play Application A- http://www.example1. 00:26:16.01 TriviaGame com/data1/ Time: 00:42:12.29- Play Application B-http://www.example2. 00:60:00.30 Shopping com/data2/ Application N/AMute Application C- http://www.example3. Plot Summary com/data3/ Display

By performing the remaining steps of the method of FIG. 5 , which arefurther described below, using the example content synchronization dataillustrated in the table above, for instance, the user's mobile devicethus may present “Application A—Trivia Game” as a supplementalexperience during the corresponding times of the program indicated inthe table (e.g., while the heartbeat message(s) indicate the playbackposition is within the time period indicated in the table). In addition,prior to and/or while presenting this supplemental experience, theuser's mobile device may access, retrieve, and/or store informationdefining the supplemental experience from the corresponding data sourcedesignated in the content synchronization data (e.g.,http://www.example1.com/data1/). Similarly, the user's computing devicemay present “Application B—Shopping Application” as a supplementalexperience during the corresponding times indicated in the tablecorresponding that supplemental experience (e.g., as based on theplayback position of the program indicated in the heartbeat message(s)),and may access, retrieve, and/or store information from thecorresponding data source (e.g., http://www.example2.com/data2/).Additionally or alternatively, when the heartbeat message indicates thatthe user has muted the sound on the content receiver device, forinstance, the user's mobile device may present “Application C—PlotSummary Display” as a supplemental experience, and may access, retrieve,and/or store information from the corresponding data source indicated inthe content synchronization data (e.g., http://www.example3.com/data3/).

Other supplemental experiences may similarly be caused to be displayed.For example, when the heartbeat message indicates that the user hasmuted the sound on the content receiver device, other types ofsupplemental experiences, such as closed-captioning and/or translationapplications may similarly be executed and/or displayed. In anotherexample, during a fast-forward or rewind operation, a supplementalexperience may cause advertisements to be displayed on the user's mobiledevice, where such advertisements reflect and/or otherwise correspond toadvertisements that were skipped in the program content via thefast-forward or rewind operation. Additionally or alternatively, duringa fast-forward or rewind operation, a supplemental experience maydisplay “catch-up” information, such as program summaries, showoutlines, and/or the like.

Referring again to FIG. 5 , after requesting the content synchronizationdata in step 506, the user's mobile device may receive the requestedcontent synchronization data in step 507. As discussed above, thecontent synchronization data may be a data table in which one or moresupplemental experiences are defined with respect to one or more timeperiods and/or events associated with playback of the correspondingprogram.

According to one or more aspects, different content synchronization datamay be provided to different users with respect to the same programmingcontent, such that even if two users, each with their own user device,were viewing the same program in the same location and receiving thesame heartbeat message from the same content receiver device, the twousers' devices might present different supplemental experiences. In somearrangements, different content synchronization data may be providedbased on capabilities of the user's device, and this may result in userswho are viewing the same program receiving different contentsynchronization data. For example, one user may be using a laptopcomputer and thus may receive first content synchronization data thatdefines a first supplemental experience that is optimized forpresentation on the laptop computer, while a second user may be using atouchscreen tablet and thus may receive second content synchronizationdata that defines a second supplemental experience that is optimized forpresentation on the touchscreen tablet (e.g., because it includescontent that is designed to be interacted with via touch-based userinput). In other arrangements, different content synchronization datamay be provided to different users based on user profile informationstored on the different user devices and/or maintained by the serviceprovider. For example, such user profile information may includeinformation about the age of each user, and different contentsynchronization data (and corresponding supplemental experiences) may beprovided to users of different ages. As noted above, even thoughdifferent viewers may be watching the same programming at differentpoints in time, the viewer's perception of the supplemental experiences(including the viewer's perception of user-generated and/oruser-submitted content associated with supplemental experiences, such asvotes cast in a poll, comments posted to a message service, etc.) may besynchronized with playback of the programming.

In step 508, the user's mobile device may compare information from themost recent heartbeat message to information in the contentsynchronization data, and in step 509, the user's mobile device maydetermine whether any of the conditions defined in the contentsynchronization data match current playback conditions of the program,as indicated by the information from the heartbeat message. For example,the user's mobile device may determine whether the heartbeat messageindicates that the program's playback position is within a time periodfor which, according to the content synchronization data, a particularsupplemental experience is to be displayed. Additionally oralternatively, the user's mobile device may determine whether theheartbeat message indicates that a particular event has occurred and/orthat the device is in a particular state (e.g., sound volume muted) forwhich, according to the content synchronization data, a particularsupplemental experience is to be displayed.

Subsequently, if it is determined, in step 508, based on the informationfrom the heartbeat message, that one or more conditions defined in thecontent synchronization data match one or more current playbackconditions, then the supplemental experiences corresponding to thosematching conditions may be presented, e.g., by executing instructionsspecified by the content synchronization data. Considering the examplecontent synchronization data in the table above, for instance, if theheartbeat message were to indicate that the sound volume has been muted,then the user's mobile device may execute instructions specified by thecontent synchronization data by loading the “Application C” supplementalexperience from the data source indicated in the table (e.g.,http://www.example3.com/data3/). While the example contentsynchronization data illustrated above includes references in the formof uniform resource locators (URLs) to data sources other than thecontent synchronization data in which corresponding supplementalexperiences are defined, the program code for providing one or moresupplemental experiences may, in some arrangements, be stored in thecontent synchronization data itself.

Having presented the supplemental experience and/or executed theinstruction specified by the content synchronization data, or if it isdetermined that the current playback information does not match anyconditions defined by the content synchronization data, the method mayreturn to step 503, and the user's mobile device again may determinewhether a heartbeat message has been received. The method then maycontinue in a loop, with the user's mobile device determining whethernew or different programs are being played and/or whether one or moreconditions of content synchronization data are met by playbackinformation specified in continuously received heartbeat messages,until, for instance, the user's mobile device is powered off and/or thesoftware application is exited.

FIG. 6 illustrates example user interfaces by which a synchronoussupplemental experience may be provided according to one or more aspectsdescribed herein. For example, user interface 601 may include videoprogramming, such as a live football game, and may be displayed (orcaused to be displayed) by a content receiver device. User interface 602may be displayed by a device paired to the content receiver device(e.g., the user's mobile device) and may represent a supplementalexperience which corresponds to content being displayed by the contentreceiver device. In the illustration in FIG. 6 , for example, userinterface 602 shows an example of an interactive polling application. Inaddition to including dynamically updated content 603 related to thevideo program (e.g., the score of the football game being displayed atthe content receiver device via user interface 601), the interactivepolling application of user interface 602 may include interactivecontrols 604 and 605 that may allow the user to provide user input (inthis case, in the form of a vote) that may sent back to a server at theservice provider's central office 103 (e.g., application server 107) foradditional processing, and that further may allow the user to provideuser input to change the content being received and/or displayed by thecontent receiver device.

Aspects of the disclosure have been described in terms of illustrativeembodiments thereof. While illustrative systems and methods as describedherein embodying various aspects of the present disclosure are shown, itwill be understood by those skilled in the art, that the disclosure isnot limited to these embodiments. Modifications may be made by thoseskilled in the art, particularly in light of the foregoing teachings.For example, each of the features of the aforementioned illustrativeexamples may be utilized alone or in combination or subcombination withelements of the other examples. For example, any of the above describedsystems and methods or parts thereof may be combined with the othermethods and systems or parts thereof described above. For example, thesteps illustrated in the illustrative figures may be performed in otherthan the recited order, and one or more steps illustrated may beoptional in accordance with aspects of the disclosure. It will also beappreciated and understood that modifications may be made withoutdeparting from the true spirit and scope of the present disclosure. Thedescription is thus to be regarded as illustrative instead ofrestrictive on the present disclosure.

The invention claimed is:
 1. An apparatus comprising: one or moreprocessors; and non-transitory memory storing computer-readableinstructions that, when executed by the one or more processors,configure the apparatus to: receive, via a first wireless protocol andfrom a content consumption device, a synchronization signal for a firstprogram being presented by the content consumption device, wherein thesynchronization signal comprises information corresponding to the firstprogram; send, via a second wireless protocol and to a computing device,a request for content synchronization data for the first program;receive, via the second wireless protocol and from the computing device,the content synchronization data for the first program; provide, basedon the content synchronization data, at least one supplemental userexperience in synchronization with the first program being presented bythe content consumption device; receive a user input associated with theat least one supplemental user experience; after the receiving the userinput, transmit, via the first wireless protocol and to the contentconsumption device, a reverse synchronization signal comprising: anidentifier of the at least one supplemental user experience, andinformation corresponding to the user input; and cause, based on theinformation corresponding to the user input, the content consumptiondevice to display a second program that is synchronized with the atleast one supplemental user experience being presented by the apparatus.2. The apparatus of claim 1, wherein the instructions, when executed bythe one or more processors, further configure the apparatus to:transmit, to at least one additional user device remote from theapparatus, the synchronization signal.
 3. The apparatus of claim 1,wherein the instructions, when executed by the one or more processors,configure the apparatus to receive the synchronization signal byreceiving a heartbeat message indicative of at least one of: a channelidentifier indicating a channel currently being displayed by the contentconsumption device, a speed identifier indicating a current playbackspeed, a permission identifier indicating a current user permissionlevel, or a volume identifier indicating a current playback volumelevel.
 4. The apparatus of claim 1, wherein the instructions, whenexecuted by the one or more processors, further configure the apparatusto: determine, based on a second synchronization signal, that the secondprogram has ended and a third program is currently being presented;access, via the second wireless protocol, second content synchronizationdata for the third program; determine, based on the second contentsynchronization data, that a second supplemental user experience isrelated to the third program; and provide the second supplemental userexperience in synchronization with the third program currently beingpresented by the content consumption device.
 5. The apparatus of claim1, wherein the instructions, when executed by the one or moreprocessors, further configure the apparatus to: determine that the atleast one supplemental user experience is customized for a user of theapparatus; and present the at least one supplemental user experiencebased on a usage of the apparatus.
 6. The apparatus of claim 1, whereinthe instructions, when executed by the one or more processors, furtherconfigure the apparatus to: transmit, via the first wireless protocoland to the content consumption device, a second reverse synchronizationsignal; and cause the content consumption device to present previouslyrecorded content based on the second reverse synchronization signal. 7.The apparatus of claim 1, wherein the reverse synchronization signalfurther comprises a request for on-demand content, and wherein thecausing the content consumption device to display the second program isin response to the request for the on-demand content.
 8. The apparatusof claim 1, wherein the reverse synchronization signal further comprisesa channel change request, and wherein the causing the contentconsumption device to display the second program is in response to thechannel change request.
 9. The apparatus of claim 1, wherein theinstructions, when executed by the one or more processors, furtherconfigure the apparatus to: after the receiving the user input, send,via the second wireless protocol and to the computing device, theinformation corresponding to the user input.
 10. A method comprising:transmitting, to a first user device local to a content consumptiondevice and via a first wireless protocol, a synchronization signalcomprising: an identifier of a first content item being presented by thecontent consumption device, and a time value indicating a current pointin a timeline of the first content item; receiving, via the firstwireless protocol and from the first user device, a reversesynchronization signal comprising an identifier of at least onesupplemental user experience being provided by the first user device insynchronization with the first content item; determining, based on theidentifier in the reverse synchronization signal, that the at least onesupplemental user experience is configured for synchronized presentationwith a second content item; and presenting, in synchronization with theat least one supplemental user experience being provided by the firstuser device, the second content item.
 11. The method of claim 10,wherein the synchronization signal is a heartbeat message comprising atleast one of: a channel identifier indicating a channel currently beingdisplayed, a speed identifier indicating a current playback speed, apermission identifier indicating a current user permission level, or avolume identifier indicating a current playback volume level.
 12. Themethod of claim 10, further comprising: accessing second contentsynchronization data; and transmitting, based on the second contentsynchronization data, a second synchronization signal for the secondcontent item to the first user device and at least one additional userdevice.
 13. The method of claim 10, further comprising: receivingnotifications corresponding to the at least one supplemental userexperience; and causing display of the received notifications.
 14. Themethod of claim 10, further comprising: determining that the at leastone supplemental user experience is associated with the second contentitem based on the identifier in the reverse synchronization signalindicating a channel change, wherein the first content item correspondsto a first channel and the second content item corresponds to a secondchannel.
 15. An apparatus comprising: one or more processors; andnon-transitory memory storing computer-readable instructions that, whenexecuted by the one or more processors, configure the apparatus to:transmit, via a first wireless protocol and to a computing device, arequest for content synchronization data associated with a first programbeing presented by a content consumption device; receive, from thecomputing device and via the first wireless protocol, the contentsynchronization data associated with the first program; determine, basedon the content synchronization data and based on a current timeline ofthe first program, a supplemental user experience associated with thefirst program; present the supplemental user experience insynchronization with the first program being presented by the contentconsumption device; send, to the content consumption device and via asecond wireless protocol, a reverse synchronization signal, the reversesynchronization signal comprising an identifier of at least oneadditional supplemental user experience being presented by the apparatusand an indication to change the first program being presented by thecontent consumption device; and cause, based on the identifier in thereverse synchronization signal and the indication to change the firstprogram, the content consumption device to present a second program insynchronization with the at least one additional supplemental userexperience being presented by the apparatus.
 16. The apparatus of claim15, wherein the instructions, when executed by the one or moreprocessors, further configure the apparatus to: prior to the receivingthe content synchronization data associated with the first program,transmit user information to the computing device, wherein the receivedcontent synchronization data is selected based on the user information.17. The apparatus of claim 15, wherein the instructions, by the one ormore processors, further configure the apparatus to: receive, from thecontent consumption device, via the second wireless protocol, and atregular intervals, a synchronization signal indicative of the firstprogram being presented by the content consumption device, wherein thesynchronization signal is a heartbeat message comprising at least oneof: a channel identifier indicating a channel currently being displayed,a speed identifier indicating a current playback speed, a permissionidentifier indicating a current user permission level, or a volumeidentifier indicating a current playback volume level.
 18. The apparatusof claim 15, wherein the second program corresponds to on-demand contentand the instructions, when executed by the one or more processors,further configure the apparatus to: transmit, via the first wirelessprotocol and to the computing device, a second request containinginformation indicating the on-demand content; receive, via the firstwireless protocol and from the computing device, second contentsynchronization data for the on-demand content indicative of a secondsupplemental user experience; and present, based on the second contentsynchronization data, the second supplemental user experience insynchronization with the on-demand content being presented by thecontent consumption device.
 19. The apparatus of claim 15, wherein theinstructions, when executed by the one or more processors, furtherconfigure the apparatus to: transmit, to the computing device, theidentifier of the at least one additional supplemental user experience;and receive, from the computing device, information for the secondprogram.